fix: Invalid UTF-8 commit messages in JSON API responses by bircni · Pull Request #37542 · go-gitea/gitea

bircni · 2026-05-04T21:21:41Z

Fix commit message serialization for API responses when a repository contains legacy commits with invalid UTF-8.

wxiaoguang · 2026-05-04T21:54:29Z

Are you sure all callers just call Message() but not read CommitMessage?

wxiaoguang · 2026-05-07T06:52:28Z

Let's do a deep clean up for the legacy problem. Otherwise, the more patches we use, the more difficulty to fix the FIXME in the future.

I will fix all regressions in first time (if any)

wxiaoguang · 2026-05-07T07:12:18Z

Partially backport: fix: Invalid UTF-8 commit messages in JSON API responses #37585

Copilot

Pull request overview

This PR addresses failures when serializing commit (and tag) messages into JSON API responses for repositories containing legacy commits with invalid UTF-8 (Fixes #37534). It introduces a dedicated commit-message representation that can provide UTF-8-safe variants, and updates API/web rendering paths to use those safe variants.

Changes:

Introduces git.CommitMessage with helpers (MessageUTF8, MessageTitle, MessageBody) and wires it into git.Commit/git.Tag.
Updates API conversion/services code paths to use UTF-8-safe commit/tag messages (and titles) instead of raw commit messages.
Updates multiple templates/feeds to display commit titles/bodies via the new helpers and removes the IsMultilineCommitMessage template helper.

Reviewed changes

Copilot reviewed 47 out of 47 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
templates/user/dashboard/feeds.tmpl	Uses explicit range var; renders push commit messages consistently.
templates/repo/view_list.tmpl	Uses UTF-8-safe commit message in repo file list.
templates/repo/settings/lfs_file_find.tmpl	Uses explicit range var; keeps summary rendering consistent.
templates/repo/latest_commit.tmpl	Uses `MessageTitle/MessageBody/MessageUTF8` for latest commit display.
templates/repo/commits_list.tmpl	Uses `MessageTitle/MessageBody/MessageUTF8` throughout commit list.
templates/repo/commits_list_small.tmpl	Uses `MessageTitle/MessageBody/MessageUTF8` in compact commit list.
templates/repo/commit_page.tmpl	Uses `MessageTitle/MessageBody/MessageUTF8` on commit page header/body.
templates/mail/repo/issue/default.tmpl	Uses `MessageTitle` in push email commit list.
templates/base/head_opengraph.tmpl	Uses `Commit.MessageBody` for OpenGraph description.
services/repository/push.go	Uses UTF-8-safe tag message for release title/note extraction.
services/repository/files/file.go	Uses UTF-8-safe commit message in file commit API response.
services/repository/files/content.go	Uses UTF-8-safe last commit message in contents API response.
services/repository/branch.go	Stores branch `CommitMessage` from `MessageTitle` instead of summary.
services/pull/pull.go	Uses UTF-8-safe commit message when building squash merge messages; uses `MessageTitle` for PR commit summaries.
services/convert/wiki.go	Uses UTF-8-safe commit message for wiki commit API.
services/convert/git_commit.go	Uses UTF-8-safe commit message in payload/API commit conversions.
services/convert/convert.go	Uses UTF-8-safe tag messages in tag API conversions.
services/agit/agit.go	Uses `MessageTitle/MessageBody` for AGit PR title/body inference.
services/actions/workflow.go	Uses `MessageTitle` for action run titles.
services/actions/notifier_helper.go	Uses raw message for skip checks; uses `MessageTitle` for workflow/schedule titles.
routers/web/repo/setting/webhook.go	Adjusts fake commit construction and uses UTF-8-safe message in webhook test payload.
routers/web/repo/editor_cherry_pick.go	Uses `MessageTitle/MessageBody/MessageUTF8` for cherry-pick/revert editor defaults.
routers/web/repo/compare.go	Uses `MessageTitle/MessageBody` for new PR title/body auto-fill from commits.
routers/web/repo/commit.go	Uses `MessageTitle` for commit diff page `<title>`.
routers/web/repo/blame.go	Uses UTF-8-safe message for blame row commit message.
routers/web/feed/file.go	Uses `MessageTitle/MessageUTF8` for file feed title/description/content.
routers/web/feed/branch.go	Uses `MessageTitle/MessageUTF8` for branch feed title/description/content.
modules/templates/util_render.go	Simplifies commit-body extraction logic used by templates.
modules/templates/util_misc.go	Removes `isMultilineCommitMessage` helper.
modules/templates/helper.go	Removes `IsMultilineCommitMessage` from template func map.
modules/repository/commits.go	Uses UTF-8-safe commit message when building push commit payloads.
modules/repository/commits_test.go	Updates tests for new `CommitMessage` struct usage.
modules/repository/branch.go	Stores branch `CommitMessage` from `MessageTitle`.
modules/git/tag.go	Embeds `CommitMessage` in `Tag` and stores raw tag message in `MessageRaw`.
modules/git/tag_test.go	Updates tag parsing tests for `CommitMessage{MessageRaw: ...}`.
modules/git/repo_tag.go	Stores raw tag message in `MessageRaw` and adjusts signature stripping.
modules/git/repo_tag_test.go	Updates tag ref parsing tests for embedded `CommitMessage`.
modules/git/repo_tag_nogogit.go	Uses embedded `CommitMessage` when creating lightweight tag objects.
modules/git/repo_tag_gogit.go	Uses embedded `CommitMessage` for lightweight/annotated tags in gogit mode.
modules/git/pipeline/lfs_nogogit.go	Uses `MessageTitle` for LFS find results’ summaries.
modules/git/commit.go	Introduces `CommitMessage` struct and UTF-8/title/body helpers; embeds in `Commit`.
modules/git/commit_test.go	Updates commit-reader tests and adds a UTF-8 sanitization test for commit messages.
modules/git/commit_sha256_test.go	Updates commit-reader SHA256 tests for new commit message storage.
modules/git/commit_reader.go	Stores commit message into `MessageRaw` instead of the old string field.
modules/git/commit_convert_gogit.go	Wraps go-git commit message into `CommitMessage{MessageRaw: ...}`.
models/git/branch.go	Stores branch commit message from `MessageTitle` in DB updates.
models/git/branch_test.go	Updates branch tests to construct commits using `CommitMessage{MessageRaw: ...}`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…7585) Backport #37542 Co-authored-by: Nicolas <bircni@icloud.com> --------- Co-authored-by: Nicolas <bircni@icloud.com>

* origin/main: (41 commits) refactor: only reset a database table when the table's data was changed (go-gitea#37573) refactor: lint bare `fill`/`stroke` colors, add vars for git graph color series (go-gitea#37543) fix(actions): report individual step status in workflow job API response (go-gitea#37592) ci: allow `chore` type in PR title lint (go-gitea#37575) refactor: remove obsolete export (go-gitea#37579) fix: Invalid UTF-8 commit messages in JSON API responses (go-gitea#37542) fix: use consistent GetUser family functions (go-gitea#37553) fix(api): return 409 message instead of empty JSON for wrong commit id (go-gitea#37572) fix(actions): prevent panic when workflow contains null jobs (go-gitea#37570) [skip ci] Updated translations via Crowdin refactor: use modernc sqlite driver as default (go-gitea#37562) refactor(templates): remove ctxData from tmpl files, use ctx.RootData instead (go-gitea#37567) ci: increase renovate frequency and fix RENOVATE_ALLOWED_POST_UPGRADE_COMMANDS (go-gitea#37565) fix(deps): update go dependencies (go-gitea#37541) refactor(deps): migrate from `nektos/act` fork to `gitea/runner` (go-gitea#37557) ci: lint PR titles with commitlint (go-gitea#37498) Make ServeSetHeaders default to download attachment if filename exists (go-gitea#37552) fix(actions): validate workflow param to prevent 500 error (go-gitea#37546) Fix various problems (go-gitea#37547) docs: fix 4 typos in CHANGELOG.md (go-gitea#37549) ... # Conflicts: # templates/repo/actions/runs_list.tmpl # templates/repo/issue/view_content/pull_merge_box.tmpl # templates/repo/pulls/status.tmpl

* origin/main: chore(renovate): detect Makefile tools, enable `go` bumps (go-gitea#37595) refactor: only reset a database table when the table's data was changed (go-gitea#37573) refactor: lint bare `fill`/`stroke` colors, add vars for git graph color series (go-gitea#37543) fix(actions): report individual step status in workflow job API response (go-gitea#37592) ci: allow `chore` type in PR title lint (go-gitea#37575) refactor: remove obsolete export (go-gitea#37579) fix: Invalid UTF-8 commit messages in JSON API responses (go-gitea#37542) fix: use consistent GetUser family functions (go-gitea#37553) fix(api): return 409 message instead of empty JSON for wrong commit id (go-gitea#37572) fix(actions): prevent panic when workflow contains null jobs (go-gitea#37570) [skip ci] Updated translations via Crowdin # Conflicts: # services/pull/pull.go

* main: chore(deps): bump tool deps and pin, update `golangci-lint` (go-gitea#37574) build: update pnpm to v11 (go-gitea#37591) chore(renovate): detect Makefile tools, enable `go` bumps (go-gitea#37595) refactor: only reset a database table when the table's data was changed (go-gitea#37573) refactor: lint bare `fill`/`stroke` colors, add vars for git graph color series (go-gitea#37543) fix(actions): report individual step status in workflow job API response (go-gitea#37592) ci: allow `chore` type in PR title lint (go-gitea#37575) refactor: remove obsolete export (go-gitea#37579) fix: Invalid UTF-8 commit messages in JSON API responses (go-gitea#37542) fix: use consistent GetUser family functions (go-gitea#37553) fix(api): return 409 message instead of empty JSON for wrong commit id (go-gitea#37572) fix(actions): prevent panic when workflow contains null jobs (go-gitea#37570) [skip ci] Updated translations via Crowdin refactor: use modernc sqlite driver as default (go-gitea#37562) refactor(templates): remove ctxData from tmpl files, use ctx.RootData instead (go-gitea#37567) ci: increase renovate frequency and fix RENOVATE_ALLOWED_POST_UPGRADE_COMMANDS (go-gitea#37565) # Conflicts: # models/user/user.go

* origin/main: (82 commits) build(deps): bump fast-uri from 3.1.0 to 3.1.2 (go-gitea#37616) fix: make clone URL respect public URL detection setting (go-gitea#37615) chore(deps): bump go-git/go-git/v5 to 5.19.0 (go-gitea#37608) chore(deps): update action dependencies (go-gitea#37603) fix(actions): fix blank lines after `::endgroup::` (go-gitea#37597) fix: treat email addresses case-insensitively (go-gitea#37600) fix(git): Fix smart http request scope bug (go-gitea#37583) chore(deps): update dependency go to v1.26.3 (go-gitea#37601) chore(deps): bump tool deps and pin, update `golangci-lint` (go-gitea#37574) build: update pnpm to v11 (go-gitea#37591) chore(renovate): detect Makefile tools, enable `go` bumps (go-gitea#37595) refactor: only reset a database table when the table's data was changed (go-gitea#37573) refactor: lint bare `fill`/`stroke` colors, add vars for git graph color series (go-gitea#37543) fix(actions): report individual step status in workflow job API response (go-gitea#37592) ci: allow `chore` type in PR title lint (go-gitea#37575) refactor: remove obsolete export (go-gitea#37579) fix: Invalid UTF-8 commit messages in JSON API responses (go-gitea#37542) fix: use consistent GetUser family functions (go-gitea#37553) fix(api): return 409 message instead of empty JSON for wrong commit id (go-gitea#37572) fix(actions): prevent panic when workflow contains null jobs (go-gitea#37570) ... # Conflicts: # tests/e2e/events.test.ts # tests/integration/eventsource_test.go

This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [code.gitea.io/gitea](https://github.com/go-gitea/gitea) | `v1.26.1` → `v1.26.2` | ![age](https://developer.mend.io/api/mc/badges/age/go/code.gitea.io%2fgitea/v1.26.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/code.gitea.io%2fgitea/v1.26.1/v1.26.2?slim=true) | --- ### Release Notes <details> <summary>go-gitea/gitea (code.gitea.io/gitea)</summary> ### [`v1.26.2`](https://github.com/go-gitea/gitea/releases/tag/v1.26.2) [Compare Source](go-gitea/gitea@v1.26.1...v1.26.2) - SECURITY - fix(permissions): Fix reading permission ([#37769](go-gitea/gitea#37769)) - fix(actions): make artifact signature payloads unambiguous ([#37707](go-gitea/gitea#37707)) - fix: Unify public-only token filtering in API queries and repo access checks ([#37118](go-gitea/gitea#37118)) - fix: Add missed token scope checking ([#37735](go-gitea/gitea#37735)) - fix(oauth): bind token exchanges to the original client request ([#37704](go-gitea/gitea#37704)) - fix(oauth): strengthen PKCE validation and refresh token replay protection ([#37706](go-gitea/gitea#37706)) - fix(web): enforce token scopes on raw, media, and attachment downloads ([#37698](go-gitea/gitea#37698)) - fix(security): enforce wiki git writes and LFS token access at request time ([#37695](go-gitea/gitea#37695)) - feat(api): encrypt AWS creds ([#37679](go-gitea/gitea#37679)) - fix(deps): update dependency mermaid to v11.15.0 \[security], add e2e test - fix(packages): Add label for private and internal package and fix composor package source permission check ([#37610](go-gitea/gitea#37610)) - fix(git): Fix smart http request scope bug ([#37583](go-gitea/gitea#37583)) - Fix basic auth bug ([#37503](go-gitea/gitea#37503)) - Fix allow maintainer edit permission check ([#37479](go-gitea/gitea#37479)) ([#37484](go-gitea/gitea#37484)) - Fix URL sanitization to handle schemeless credentials ([#37440](go-gitea/gitea#37440)) ([#37471](go-gitea/gitea#37471)) - Fix attachment Content-Security-Policy ([#37455](go-gitea/gitea#37455)) ([#37464](go-gitea/gitea#37464)) - chore(deps): bump go-git/go-git/v5 to 5.19.0 ([#37608](go-gitea/gitea#37608)) - BUGFIXES - fix(pull): handle empty pull request files view to allow reviews ([#37783](go-gitea/gitea#37783)) - fix(markup): make RenderString never fail ([#37779](go-gitea/gitea#37779)) - fix: add natural sort to sortTreeViewNodes ([#37772](go-gitea/gitea#37772)) - fix: package creation unique conflict ([#37774](go-gitea/gitea#37774)) - fix!: add DEFAULT\_TITLE\_SOURCE setting for pull request title default behavior ([#37465](go-gitea/gitea#37465)) - fix: Allow direct commits for unprotected files with push restrictions ([#37657](go-gitea/gitea#37657)) - fix(actions): wrong assumption that run id always >= job id ([#37737](go-gitea/gitea#37737)) - fix(auth): set User-Agent on avatar fetch and sync avatar on link-account register ([#37564](go-gitea/gitea#37564)) ([#37588](go-gitea/gitea#37588)) - fix(actions): deadlock between PrepareRunAndInsert and UpdateTaskByState ([#37692](go-gitea/gitea#37692)) - fix(repo): /generate must sync the branch table for the new repo ([#37693](go-gitea/gitea#37693)) - build: Fix snap build (1.26) - fix(actions): run TransferLogs on UpdateLog{Rows:\[], NoMore:true} ([#37631](go-gitea/gitea#37631)) - fix show correct mergebase - fix: make clone URL respect public URL detection setting ([#37615](go-gitea/gitea#37615)) - fix: "run as root" check ([#37622](go-gitea/gitea#37622)) - chore(deps): update dependency go to v1.26.3 ([#37601](go-gitea/gitea#37601)) - Compare dropdown fails when selecting branch with no common merge-base ([#37470](go-gitea/gitea#37470)) - fix: treat email addresses case-insensitively ([#37600](go-gitea/gitea#37600)) - fix(actions): fix blank lines after ::endgroup:: ([#37597](go-gitea/gitea#37597)) - fix(actions): report individual step status in workflow job API response ([#37592](go-gitea/gitea#37592)) - fix: Invalid UTF-8 commit messages in JSON API responses ([#37542](go-gitea/gitea#37542)) - fix: use consistent GetUser family functions ([#37553](go-gitea/gitea#37553)) - fix(api): return 409 message instead of empty JSON for wrong commit id ([#37572](go-gitea/gitea#37572)) - fix(actions): prevent panic when workflow contains null jobs ([#37570](go-gitea/gitea#37570)) - Make ServeSetHeaders default to download attachment if filename exists ([#37552](go-gitea/gitea#37552)) ([#37555](go-gitea/gitea#37555)) - Fix(actions): validate workflow param to prevent 500 error ([#37546](go-gitea/gitea#37546)) ([#37554](go-gitea/gitea#37554)) - Don't unblock run-level-concurrency-blocked runs in the resolver ([#37461](go-gitea/gitea#37461)) ([#37538](go-gitea/gitea#37538)) - Fix(packages): use file names for generic web downloads ([#37514](go-gitea/gitea#37514)) ([#37520](go-gitea/gitea#37520)) - Fix merge autodetect can't close other PRs but only the last one when multiple PRs are pushed at once ([#37512](go-gitea/gitea#37512)) ([#37516](go-gitea/gitea#37516)) - Fix update branch protection order ([#37508](go-gitea/gitea#37508)) ([#37513](go-gitea/gitea#37513)) - Fix mCaptcha broken after Vite migration ([#37492](go-gitea/gitea#37492)) ([#37509](go-gitea/gitea#37509)) - Fix review submission from single-commit PR view ([#37475](go-gitea/gitea#37475)) ([#37485](go-gitea/gitea#37485)) - Fix scheduled action panic with null event payload ([#37459](go-gitea/gitea#37459)) ([#37466](go-gitea/gitea#37466)) - Make GetPossibleUserByID can handle deleted user ([#37430](go-gitea/gitea#37430)) ([#37431](go-gitea/gitea#37431)) - Remove excessive quote from terraform instructions ([#37424](go-gitea/gitea#37424)) ([#37426](go-gitea/gitea#37426)) - Fix color regressions, add `priority` color ([#37417](go-gitea/gitea#37417)) ([#37421](go-gitea/gitea#37421)) - MISC - Add CurrentURL template variable back ([#37444](go-gitea/gitea#37444)) ([#37449](go-gitea/gitea#37449)) Instances on **[Gitea Cloud](https://cloud.gitea.com)** will be automatically upgraded to this version during the specified maintenance window. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.erwanleboucher.dev/eleboucher/apoci/pulls/47

fix

d1873b6

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label May 4, 2026

wxiaoguang marked this pull request as draft May 5, 2026 05:42

wxiaoguang force-pushed the fix/utf8-encoding branch from f9d4715 to 71d051f Compare May 7, 2026 06:59

wxiaoguang added backport/done All backports for this PR have been created backport/manual No power to the bots! Create your backport yourself! backport/v1.26 This PR should be backported to Gitea 1.26 labels May 7, 2026

wxiaoguang marked this pull request as ready for review May 7, 2026 07:00

wxiaoguang mentioned this pull request May 7, 2026

fix: Invalid UTF-8 commit messages in JSON API responses (#37542) #37585

Merged

wxiaoguang force-pushed the fix/utf8-encoding branch from 71d051f to 1a142f0 Compare May 7, 2026 07:11

wxiaoguang added this to the 1.27.0 milestone May 7, 2026

wxiaoguang added the type/bug label May 7, 2026

wxiaoguang force-pushed the fix/utf8-encoding branch from 1a142f0 to 9203d7c Compare May 7, 2026 07:25

wxiaoguang requested a review from Copilot May 7, 2026 07:25

Copilot started reviewing on behalf of wxiaoguang May 7, 2026 07:26 View session

refactor

1b0ef0d

wxiaoguang force-pushed the fix/utf8-encoding branch from 9203d7c to 1b0ef0d Compare May 7, 2026 07:30

Merge branch 'main' into fix/utf8-encoding

56624fe

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread modules/git/commit.go

Comment thread modules/git/repo_tag_gogit.go

Comment thread modules/git/commit_test.go

wxiaoguang added 2 commits May 7, 2026 15:40

fix test

ab06f48

fix test

21408e0

wxiaoguang approved these changes May 7, 2026 •

edited

Loading

View reviewed changes

GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels May 7, 2026

wxiaoguang added 2 commits May 7, 2026 16:50

fix incorrect tag message parser

f33cccb

fix test

fe77f96

wxiaoguang force-pushed the fix/utf8-encoding branch from db5c5b6 to fe77f96 Compare May 7, 2026 09:20

lunny approved these changes May 7, 2026

View reviewed changes

GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels May 7, 2026

bircni merged commit c9b9e37 into go-gitea:main May 7, 2026
23 checks passed

silverwind pushed a commit that referenced this pull request May 7, 2026

fix: Invalid UTF-8 commit messages in JSON API responses (#37542) (#3…

3004c45

…7585) Backport #37542 Co-authored-by: Nicolas <bircni@icloud.com> --------- Co-authored-by: Nicolas <bircni@icloud.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: Invalid UTF-8 commit messages in JSON API responses#37542

fix: Invalid UTF-8 commit messages in JSON API responses#37542
bircni merged 7 commits into
go-gitea:mainfrom
bircni:fix/utf8-encoding

bircni commented May 4, 2026

Uh oh!

wxiaoguang commented May 4, 2026

Uh oh!

wxiaoguang commented May 7, 2026

Uh oh!

wxiaoguang commented May 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

bircni commented May 4, 2026

Uh oh!

wxiaoguang commented May 4, 2026

Uh oh!

wxiaoguang commented May 7, 2026

Uh oh!

wxiaoguang commented May 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants